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In 2007 Georgia instituted a flexibility policy through which school districts enter into performance contracts 
with the state, receiving waivers from state rules, provisions, and guidelines in exchange for agreeing to 
meet annual accountability targets. The performance contracts are intended to incentivize innovations that 
increase achievement among all students. Between 2008/09 and 2016/17, 178 of Georgia’s 180 districts 
entered into a performance contract. The Georgia Department of Education requested an analysis of how 
student achievement changed after districts adopted a performance contract and what factors were related 


to those changes. The department also requested information on how districts used their performance 
contract to prioritize innovations in local practice. Overall, the study found little evidence that changes in 
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changes in achievement across districts, once other factors were adjusted for. Changes in achievement were 
largely unrelated to district characteristics, including urbanicity, timing of performance contract adoption, 
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related to college and career readiness, teacher certification requirements, instructional spending, and 
funding for school improvement after adopting a performance contract. Leaders perceived broad benefits 
from the priority innovations they identified, especially for staff and school climate, but also indicated that 
waivers were not required to implement many of the innovations. Despite the perceived benefits, changes 
in achievement were largely unrelated to the academic, human resources, and financial innovations that 
district leaders reported prioritizing after adopting a performance contract. 


In 2007 Georgia education leaders and policymakers instituted a flexibility policy through which school districts 
enter into performance contracts with the state that grant waivers from state rules, provisions, and guidelines 
and allow schools and districts greater autonomy. In exchange, schools and districts must meet academic perfor- 
mance targets within five years or face consequences such as state takeover of schools.’ The performance con- 
tracts are intended to incentivize innovations in local practice that ultimately increase student achievement.” The 
waivers enable districts to use resources differently to implement innovations that align with their educational 
mission, vision, and goals for improving student outcomes. Georgia’s goal for the performance contracts is to shift 
its education system’s focus from compliance to student achievement. 


Georgia’s flexibility policy allowed districts to become Charter Systems or Strategic Waiver School Systems (SWSS; 
see box 1 for definitions of key terms used in the report). The initial policy allowed districts to become Charter 
Systems, which receive a blanket waiver covering all allowable state laws and 
regulations—the same flexibilities that individual charter schools receive. Charter For additional information, 
Systems must implement local school governance teams for each school in the including background 
district and can seek approximately $100 a year per pupil in supplementary on the study, technical 
funding to support their transformation. methods, supporting 


analyses, the survey 


1. No districts have yet faced consequences, in part because performance targets have been instrument, and the 
revised due to changes in standardized assessments and changes to Georgia’s College and interview protocol, access 
Career Ready Performance Index. the report appendixes at 


2. Innovations are enacted changes to standard academic, human resources, or financial practices https://go.usa.gov/x7GGw. 
made by a district. 
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Few school districts chose to seek charter status, leading state policymakers to amend the law in 2008 to give 
districts another option—to become an SWSS. An SWSS must request waivers from specific regulations rather 
than receiving a waiver from all possible regulations, as Charter Systems do. Across all SWSSs the state issued 
waivers from 122 regulations. On average, each SWSS requested 39 waivers, and across all SWSSs more than 
5,000 waivers were granted. SWSSs are not required to implement local school governance teams and do not 
receive supplemental per pupil funding. 


In 2015 the state required all 180 Georgia school districts to become a Charter System or SWSS or to remain as 
Title 20/No Waivers School Systems (with no performance contract). Between 2008/09 and 2016/17, 178 districts 
adopted a performance contract, with 136 initially choosing to operate as an SWSS and 42 initially choosing to 
operate as a Charter System.? Groups of districts were granted waivers and implemented performance contracts 
in each of the school years between 2008/09 and 2016/17, which means that there are nine adoption cohorts 
(one for each school year). For the analyses the first seven adoption cohorts (2008/09-2014/15) were grouped 
into a single early adoption cohort group, the 2015/16 adoption cohort was the middle adoption cohort group, 
and the 2016/17 adoption cohort was the late adoption cohort group. 


The Georgia Department of Education requested the Regional Educational Laboratory Southeast’s help in analyz- 
ing districts’ experiences under the flexibility policy. The department was specifically interested in how student 
achievement changed after districts adopted a performance contract, the district characteristics associated with 
the change in achievement, and the innovations that districts prioritized for implementation under their perfor- 
mance contract. It also was interested in the practices and perceived benefits to staff and school climate of SWSSs 
and Charter Systems where student achievement improved after the district adopted a performance contract. 


State leaders can use the study findings to produce communications, develop services, and allocate resources 
more effectively in order to help district and school efforts use the waivers in their performance contracts. The 
findings will also provide information to Georgia districts about how other districts prioritized innovations and 
experiences under the flexibility policy. The results also may be of interest to other states considering similar 
deregulation policies. Georgia’s flexibility policy is part of a growing trend in education reform to provide districts 
autonomy from traditional state-led education processes and policies in order to allow more flexibility in local 
decisionmaking, with the goal of improving student outcomes (Bulkley, 2005; Whitty & Power, 2000; Wrabel 
et al., 2016; see appendix A for a review of the literature on district and school flexibility). The results contrib- 
ute to knowledge about how districts experience flexibilities and autonomy to implement innovations that could 
inform other states’ policies and plans. 


3. In 2017/18 and 2018/19, four SWSSs chose to switch to Charter Systems. This study examines districts’ experiences following their 
initial performance contract, so those recent switches are beyond its scope. 
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Box 1. Key terms 


Changes in student achievement. The average difference in student achievement before and after performance contract adop- 
tion for districts that adopted a performance contract compared with differences in the same years for districts that did not adopt 
one. Student achievement is measured by state standardized end-of-grade assessments in grades 3—8 and end-of-course exams in 
grades 9-12. Meaningful changes are defined as changes that are statistically significant and .05 standard deviation or larger, and 
small changes are changes that are statistically significant and smaller than .05 standard deviation. These thresholds are based on 
empirical benchmarks described in appendix B. 


Charter System. A school district whose performance contract provides blanket waivers covering all allowable state laws and 
regulations—the same flexibilities that individual charter schools receive. Charter Systems must implement local school gover- 
nance teams for each school and can seek approximately $100 a year per pupil in supplementary funding to support their transfor- 
mation. Across Georgia, 42 districts initially chose Charter System status under the state’s flexibility policy. 


Early adoption cohort group. The 3 SWSSs and 28 Charter Systems in the seven cohorts that adopted a performance contract 
between 2008/09 and 2014/15, before Georgia passed an amendment requiring all districts to choose a status. 


Innovation. An enacted change to standard academic, human resources, or financial practices made by a district. 


Late adoption cohort group. The 76 SWSSs and 10 Charter Systems that adopted a performance contract in 2016/17, the last 


group to do so. 


Middle adoption cohort group. The 57 SWSSs and 4 Charter Systems that adopted a performance contract in 2015/16, immedi- 
ately after the amendment requiring all districts to choose a status. 


Strategic Waivers School System (SWSS). A school district whose performance contract requires that the district request 
waivers from specific regulations. SWSSs are not required to implement local school governance teams and do not receive supple- 
mental per pupil funding. Across Georgia, 136 districts initially chose SWSS status under the state’s flexibility policy. 


Research questions 
The study addressed three research questions, designed broadly to understand the relationship between adopt- 
ing a performance contract and changes in student achievement and the types of local practices and innovations 


that district leaders prioritized after adopting a performance contract: 


1. How did student achievement in English language arts and math change after districts adopted a performance 
contract, once other factors, including prior achievement, were adjusted for? 


a. Did changes in achievement vary across districts? 
b. Were changes in achievement related to urbanicity (urban, suburban, town, or rural locale), adoption cohort 
group (early, middle, or late), or district type (SWSS or Charter System), after other factors were adjusted 


for? 


2. What local practices and innovations did district leaders report prioritizing after adopting a performance con- 
tract, and which of these were perceived as most beneficial? 


3. Were changes in student achievement in English language arts and math related to innovations that district 
leaders reported prioritizing after adopting a performance contract, after other factors were adjusted for? 
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Understanding changes in student achievement and district practices under performance contracts 


The study team used administrative data from the Georgia Department of Education to examine how student 

achievement in English language arts and math changed from before districts adopted a performance contract to 

after, after other factors were adjusted for. Specifically, the analyses of the administrative data provide informa- 

tion on: 

e Overall changes in achievement in grade 3—8 English language arts and math, grade 9 English, and grade 9-12 
Algebra |. 

e Variability in the changes in achievement in grade 3-8 English language arts and math, grade 9 English, and 
grade 9-12 Algebra | across districts. 

e Links among district characteristics, priority innovations, and changes in achievement. 


The analyses of implementation of innovations use administrative data as well as survey and interview data that 
the Georgia Department of Education collected. The survey asked district leaders to identify the innovations that 
were their top, second, and third priorities in each of three waiver areas—academic programs, human resources, 
and finances—after performance contract adoption. The interviews allowed leaders from 10 districts with some 
of the largest improvement in student achievement after performance contract adoption to provide detailed 
responses about the innovations that they reported prioritizing in the survey and their decisionmaking processes. 
They described whether they were implementing the innovations and, if so, how. In the interviewed districts, 
most of the prioritized innovations were being implemented, although some still were being planned. 


Variables examined in the study 


The study variables included district-level student demographic composition, district type (SWSS or Charter 
System), district composition by school type (percentages of schools in each district that were traditional public, 
public charter, magnet, special education, alternative, other, and Title | status schools), urbanicity (urban, subur- 
ban, town, or rural locale), enrollment, and measures from the survey that indicate the innovations that districts 
were prioritizing for implementation. The data sources, sample, and methods used are summarized in box 2 and 
detailed in appendix B. 


Box 2. Data sources, sample, and methods 


Data sources. For research question 1 the study team used data from Georgia Department of Education administrative records. 
These data were supplemented with school data from the U.S. Department of Education’s Common Core of Data. For research 
question 2 the study team used data from a survey that the Georgia Department of Education administered to district leaders in 
the 178 Strategic Waivers School Systems (SWSSs) and Charter Systems and data from interviews that the department conducted 
with a subset of district leaders in 10 SWSSs and Charter Systems who completed the survey. For research question 3 the study 
team supplemented the data used in research question 1 with survey data on district priority innovations. 


Sample. The administrative data used for research questions 1 and 3 include all students in grades 3-12 from 2005/06-2017/18 
from the 178 SWSSs and Charter Systems in Georgia. The data include 10.5 million student scores on standardized achievement 
tests in four grade-subject groups (end-of-grade assessments in grade 3-8 English language arts and grade 3-8 math and end-of- 
course assessments in grade 9 English and grade 9-12 Algebra |), averaged by district and year. The 178 districts were categorized 
into three adoption cohort groups (early, middle, or late) by the year in which they adopted a performance contract (see box 1). 

The survey data used for research questions 2 and 3 included responses from leaders in 133 districts. The district-level survey 
response rate was 75 percent. The study team found that the districts with survey responses were representative of all districts in 
Georgia based on observable characteristics (see appendix B). 

The interview data used for research question 2 were from leaders in 10 districts that were ranked in the top 15 percent of 
districts based on changes in student achievement after performance contract adoption. Of the districts interviewed, seven were 
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SWSSs, and three were Charter Systems. The interview data are not representative of all districts; districts were selected for their 
potential to identify hypotheses about promising practices that could inform the Georgia Department of Education and other 
districts in the state. 


Methodology. For research question 1 the study team conducted a district-level longitudinal analysis of student achievement 
(that is, a generalized difference in differences analysis) to examine changes in achievement after performance contract adoption 
and the degree to which changes in achievement were related to urbanicity (urban, suburban, town, or rural locale), adoption 
cohort group (early, middle, or late), and district type (SWSS or Charter System). Because districts adopted performance contracts 
at different points in time, change is a relative measure, defined as the average difference between student achievement before 
and after performance contract adoption for districts that adopted a performance contract compared with differences in those 
same years for districts that did not adopt one. For the earliest adoption cohort (2008/09), changes in achievement are relative 
to changes in achievement in the same years among other districts that adopted a performance contract in a later year. For the 
latest adoption cohort (2016/17), changes in achievement are relative to changes in achievement in the same years among districts 
that adopted a performance contract between 2008/09 and 2015/16; the comparison can be thought of as the value-added of 
adopting a performance contract for the last adoption cohort over and above any effects observed among districts that had 
already adopted one. For districts that adopted a performance contract between 2009/10 and 2015/16, the comparison condition 
includes both changes in achievement among districts that had not yet adopted a performance contract and changes in achieve- 
ment among districts that had already adopted one (see table B4 in appendix B for more details). 

The modeling approach for research question 1 included school year and district fixed effects (that is, fixed intercepts); incor- 
porated district-level random effects for changes in student achievement after districts adopted a performance contract, which 
allowed the study team to examine how changes varied across districts; and adjusted for time-varying district compositional char- 
acteristics, including gender, race/ethnicity, free and reduced-price lunch status, individualized education program status, English 
learner status, and school type (traditional public, public charter, magnet, special education, alternative, other, and Title | status). 
Interaction terms between the performance contract adoption indicator and indicators for urbanicity, adoption cohort group, 
and district type were used to examine their relationship (that is, the degree to which those characteristics relate to changes in 
achievement after performance contract adoption). 

For research question 2 the study team used descriptive statistics to identify priority innovations related to academic pro- 
grams, human resources, and finances after districts adopted a performance contract, as indicated by survey responses from 
district leaders. Interviews with district leaders were recorded, transcribed, and qualitatively coded in NVivo using a coding struc- 
ture developed by the study team (see appendix B). The coded transcripts were analyzed for common themes related to districts’ 
decisionmaking regarding innovative practices and waiver use, their implementation of innovative practices, whether districts 
required specific waivers to implement their innovations, and their experience with Georgia’s district and school flexibility policy. 

For research question 3 the team used the same analytical approach as in research question 1 but added interaction terms 
between the performance contract adoption indicator and district leader—reported priority innovations (see appendix B). 


Findings 


The report findings are organized by research question. The first two sections focus on findings related to research 
question 1, including changes in student achievement after districts adopted a performance contract; variation in 
changes in achievement; and the relationship between changes in achievement and urbanicity, adoption cohort 
group, and district type. The next six sections focus on descriptive findings related to research question 2, includ- 
ing the innovations most frequently identified as priorities by district leaders in the statewide survey, insights into 
the motivation behind innovation priorities from interviews with district leaders, and perceived benefits from 
the innovations the districts identified in surveys and interviews. The final section focuses on findings related to 
research question 3 on the relationship between changes in student achievement and priority innovations identi- 
fied by district leaders in the statewide survey. 
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Changes in student achievement after districts adopted a performance contract were small and were 
not consistently positive or negative, but variation in change in achievement was significant and 
meaningful 


Average changes in student achievement across grades and subjects after districts adopted a performance con- 
tract ranged from —.02 to .03 standard deviation, once other factors were adjusted for (table 1). The only average 
change in achievement that was statistically significant was in grade 3—8 math (an improvement of .03 standard 
deviation, or about three weeks of learning; Hill et al., 2008). None of the average changes was meaningfully large 
(.05 standard deviation or larger). 


Variation in district changes in student achievement was significant and meaningful. In grade 3-8 math approxi- 
mately 90 percent of districts were expected to have true change in achievement between —.10 and .16 standard 
deviation (see table 1).4 This result indicates that while the statewide average change in grade 3-8 math achieve- 
ment was small (.03 standard deviation), some districts had changes that were meaningfully above the average 
and some had changes that were meaningfully below the average. In addition, the probability that achievement 
in grade 3-8 math changed by at least .05 standard deviation is approximately 40 percent.° Similar results were 
observed in achievement in grade 3-8 English language arts, grade 9 English, and Algebra | (figure 1). 


Table 1. Changes in student achievement in English language arts and math after Georgia districts adopted a 
performance contract, 2005/06-2017/18 


Change in student achievement by district 


Probability of 


Number being at least 
of districts .05 standard 
(district-by-year Average Standard 90 percent deviation 
Subject observations) (standard error) deviation? range (percent) 
English language arts (grades 3-8) 178 (2,136) .02 (.01) .09* —.13, .16 35 
Math (grades 3-8) 178 (2,136) .03* (.01) .08* —.10, .16 40 
Grade 9 English end-of-course assessment 178 (2,124) -.01 (.01) .08* —14, 11 20 
Algebra | end-of-course assessment (grades 9-12) 178 (1,863) -.02 (.02) 12* —.22, 18 27 


* Statistically significant at the .05 level or lower. 
Note: See appendix B for a full description of the methods used to generate these results and table C1 in appendix C for the full set of results. 
a. Refers to the standard deviation of district changes in achievement. 


Source: Authors’ analysis using administrative data from the Georgia Department of Education, 2005/06—2017/18. 


4. These probabilities reflect the distribution of true district changes, which assumes that the districts and outcomes in the sample 
are representative of future districts and outcomes in Georgia. Using the average and the standard deviation of change for a given 
outcome, a randomly selected district would be expected to have a 90 percent probability of a true change in achievement within the 
specified interval. 

5. This represents the expected probability that true district change in achievement is .05 standard deviation or larger. 
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Figure 1. Changes in student achievement after Georgia districts adopted a performance contract varied in 
size and direction, 2005/06—2017/18 


Grade 3-8 English language arts Grade 3-8 math 
Change in student achievement (standard deviations) Change in student achievement (standard deviations) 
0.2 | 0.2 j 
iii. eestor tl] 


: wa a = 
-0.1 


0.2 -0.2 
Grade 9 English end-of-course assessment Grade 9-12 Algebra | end-of-course assessment 
Change in student achievement (standard deviations) Change in student achievement (standard deviations) 
0.2 0.2 
0.1 0.1 


0.0 puree rel 0.0 aati 
01 


—0.2 —0.2 


Note: Each blue line represents one district; the vertical axes were truncated at +/— .2 standard deviation to improve comparability of scales across 
subjects and maximize readability. These plots were generated from the model that yielded the findings in table 1. See the methods section of appendix 
B for details on the regression models used in these analyses, and see tables C1 and C2 in appendix C for related results. 


Source: Authors’ analysis using administrative data from the Georgia Department of Education, 2005/06—2017/18. 


Changes in student achievement after districts adopted a performance contract were largely 
unrelated to district urbanicity, adoption cohort group, or district type 


There was minimal evidence of a relationship between changes in student achievement after districts adopted a 
performance contract and urbanicity, adoption cohort group, or district type (see table 2 for standardized differ- 
ences in changes between groups). For example, the change in achievement in grade 3—8 English language arts 
was about .04 standard deviation smaller for urban districts than for suburban districts, although this result is 
not statistically significant. The changes in achievement for the early adoption cohort group (2008/09-2014/15) 
were comparable to those for the middle (2015/16) and late (2016/17) adoption cohort groups, once composi- 
tional differences and trends over time were adjusted for, even though the early adoption cohort group had 
higher average student achievement across subjects before performance contract adoption. Despite differences 
in average student achievement, urban, suburban, town, and rural districts changed in mostly comparable ways 
after adopting a performance contract. However, changes in achievement in grade 9-12 Algebra | were about 
14 standard deviation smaller in urban districts than in suburban districts,® which was statistically significant. In 
addition, there was no difference in the degree to which achievement in any subject changed between districts 
that became SWSSs or Charter Systems as part of their performance contract. 


6. After performance contract-related priority innovations identified by district leaders in the survey were adjusted for, the difference 
falls to about .06 standard deviation and is no longer statistically significant. 
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Table 2. Changes in student achievement after Georgia districts adopted a performance contract and their 
relationship with urbanicity, adoption cohort group, or district type, 2005/06-2017/18 


Grades 3-8 (CTg-To( =. Pd 


English English end-of- Algebra | end-of- 
WETaE] +) (=) language arts course assessment course assessment 


Urbanicity (suburban is reference group) 


City -0.04 (0.03) -0.02 (0.03) 0.00 (0.03) -0.14 (0.05)* 
Town -0.04 (0.03) 0.01 (0.03) —0.06 (0.03) -0.07 (0.05) 
Rural —0.03 (0.02) 0.02 (0.02) -0.01 (0.02) 0.01 (0.04) 
Adoption cohort group (early [2005/06—2014/15] is reference group) 

Middle (2015/16) —0.01 (0.03) -0.02 (0.03) -0.05 (0.03) -0.01 (0.05) 
Late (2016/16) 0.01 (0.03) -0.02 (0.03) 0.00 (0.03) 0.04 (0.06) 
District type (charter is reference group) 

Strategic Waivers School System 0.00 (0.03) 0.01 (0.03) 0.02 (0.03) —0.05 (0.05) 
Number of districts 178 178 178 178 
District-by-year observations 2,136 2,136 2,124 1,863 


* Significant at the .05 level or lower. 
Note: Numbers in parentheses are standard errors. See appendix tables C1—C3 in appendix C for full model results. 


Source: Authors’ analysis using administrative data from the Georgia Department of Education, 2005/06-—2017/18. 


Innovations that enhance preparedness for college and career were the most frequently prioritized 
academic innovations after districts adopted a performance contract 


In survey responses, leaders in 29 percent of districts reported prioritizing innovations related to college and 
career academies, with 20 percent ranking college and career academies as their top priority among the 35 aca- 
demic innovations in the survey (figure 2; see figure C1 in appendix C for rankings of all academic innovations). 
Dual enrollment options were the second most frequently ranked innovations, and alternate career pathways or 
industry certifications were the third. Interview respondents often reported implementing dual enrollment and 
alternate career pathways as part of college and career academies. District leaders ranked other academic inno- 
vations, such as personalized learning, specialty classes, graduation requirements, and student promotion and 
retention policies, as a priority less frequently. 


Districts may have a variety of motivations for prioritizing college- and career-related innovations. These include 
increased community involvement and investment, desire to meet students’ needs, and accountability pressure. 
In an interview one Charter System leader described the district’s prioritizing its college and career academy 
as a community-building opportunity. The district brought together partners from the community and higher 
education to ask, “What do we need in this community? Where are we struggling to find folks that can perform, 
work in these areas?” An SWSS superintendent described a similar effort to align the district’s career academy 
and work-based internship program with community and student needs: “...especially with what we’re trying to 
develop in our community, and that’s the close relationship with the industry, we tried with our career academy. 
We did surveys; we looked at state reports to try to find out where the jobs were. So we planned our programs 
around that. We eliminated, for instance, cosmetology, because there are not that many jobs out there. But the 
intern[ship program] with the seat time [waiver] enabled us to be more targeted, more focused on getting actual 
careers for our students.” Prioritizing these innovations may also be related in part to Georgia’s college and career 
readiness accountability system, which sets performance targets for the college and career readiness index. 
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Figure 2. Innovations that enhance preparedness for college and career were the most frequently prioritized 
academic innovations after districts adopted a performance contract, according to a survey of Georgia district 
leaders, 2019 


m@ Total responses m #1 priority m #2 priority m #3 priority 


College and career academies 

Dual enrollment 

Alternative education programs 

Alternate career pathways or certifications 
Course scheduling 


Formative assessments 


College and career readiness programs 


Early identification and intervention system 


o 


10 20 30 


Percent of districts 


Note: Sample includes 133 districts. 


Source: Authors’ analysis using data from surveys of district leaders administered by the Georgia Department of Education, 2019. 


Teacher certification was the most frequently prioritized human resources innovation after districts 
adopted a performance contract 


In survey responses leaders in 71 percent of districts reported prioritizing innovations related to teacher certifica- 
tion flexibility, with 54 percent ranking teacher certification as their top priority among the 22 human resources 
innovations in the survey (figure 3; see figure C2 in appendix C for rankings of all human resources innovations). 


In interviews district leaders made several statements suggesting that they were using teacher certification 
waivers. For example, they reported hiring noncertified teachers with content expertise as part of implement- 
ing innovative programs, such as college and career academies or alternative learning programs. Interview 


Figure 3. Teacher certification was the most frequently prioritized human resources innovation after districts 
adopted a performance contract, according to a survey of Georgia district leaders, 2019 


m@ Total responses m#1priority m #2 priority m #3 priority 


Certification requirements 

Staff-to-student ratio in the classroom 

Instructional delivery model to change class size 

Number of school days, daily school hours, or school year 
Teacher recruitment practices 

Professional learning approach 


Additional summer academic programs for students 


Differentiated salary system 


0 20 40 60 80 


Percent of districts 


Note: Sample includes 133 districts. 


Source: Authors’ analysis using data from surveys of district leaders administered by the Georgia Department of Education, 2019. 
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respondents also reported hiring noncertified teachers to teach academic courses, usually in response to teacher 
shortages in specific subjects. Teacher shortages were identified as driving the need to waive teacher certification 
requirements, especially in rural areas, according to a district leader, “because we’re a smaller area and we just 
don’t have a lot to offer people ... generally it’s people who grew up here and have family here.” One leader, who 
reported finding the waiver useful “in several situations, probably over the past three or four years” for identify- 
ing “highly qualified staff, and then getting them into our school system, getting them into the right training pro- 
grams,” summarized the waiver’s benefit as follows: “Instead of worrying about certification first, we’re worrying 
about student success first, which makes a big difference.” 


According to interviewed district leaders, other circumstances that motivated prioritizing teacher certification 
innovations included filling positions that receive only one or two applicants, especially at the middle and high 
school levels; recruiting fluent Spanish-speaking teachers who can teach core content areas in Spanish for a 
dual-language immersion program; hiring content experts who had prior careers, such as photographers, pianists, 
engineers, or college English professors; and opening more pathways to employment for out-of-state licensed 
teaching professionals. 


The most frequently prioritized financial innovations after districts adopted a performance contract 
involved instructional spending and state, local, and federal funds in support of school improvement 
plans 


In survey responses leaders in 67 percent of districts reported prioritizing financial innovations related to instruc- 
tional spending, and leaders in 58 percent of districts reported prioritizing state, local, and federal funds for 
school improvement plans (figure 4; see figure C3 in appendix C for rankings of all financial innovations). Non- 
instructional spending, the next most frequently prioritized innovation, was reported as a priority by leaders in 
24 percent of districts. 


In interviews district leaders spent more time discussing academic and human resources innovations than finan- 
cial innovations, but they did comment on using financial flexibilities for innovations related to effectively allocat- 
ing funds to align with students’ needs. Interview respondents emphasized that flexibility in both instructional 


Figure 4. The most frequently prioritized financial innovations after districts adopted a performance contract 
involved instructional spending and state, local, and federal funds in support of school improvement plans, 
according to a survey of Georgia district leaders, 2019 


m@ Total responses m#1priority m #2 priority m #3 priority 


Instructional spending — | 


State, local, and federal funds a 
SSS 
fs] 


in support of school improvement ™™ 


Noninstructional spending — 


Noninstructional categorical allotment 


Categorical allotment for facilities or maintenance = 


Instructional categorical allotment 


ie) 20 40 60 80 
Percent of districts 


Note: Sample includes 133 districts. 


Source: Authors’ analysis using data from surveys of district leaders administered by the Georgia Department of Education, 2019. 
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spending and state, local, and federal funds in support of school improvement plans enabled them to allocate 
funding based on specific community, school, and student needs, including factors such as free or reduced-price 
lunch status, gifted status, and English learner status as well as for labs or pull-out services related to early inter- 
vention and remediation. In different districts aligning funding to specific needs included allowing schools to 
“orioritize those areas that, for their school community or context, are necessary supports to have in place”; eval- 
uating the budget in terms of the district’s early intervention program and remediation classes for specific grade 
levels and allocating funding to needs in those areas; and distributing funding to wraparound services to increase 
students’ readiness to learn. 


After adopting a performance contract, many districts prioritized innovations that did not require a waiver 


Many district leaders reported in the survey that a waiver was not necessary to implement the innovations that 
their districts prioritized under their performance contract. In each category of innovations, districts prioritized a 
mix of innovations that required waivers to implement and innovations that could be implemented without a waiver 
(figure 5). District leaders most frequently identified needing a waiver to implement academic innovations related 
to credit requirements and availability and enrichment or other specialty classes. The human resources and finan- 
cial innovations most frequently identified as needing a waiver included instructional delivery models to change 
class size, teacher certification requirements, differentiated salary schedules, and noninstructional and instructional 
categorical allotments. These were closely related to the “big four” waivers: class size and reporting requirements, 
teacher certification requirements, salary schedule requirements, and direct classroom expenditure control.’ 


District leaders perceived improvements across broad outcomes after they adopted a performance 
contract 


In surveys district leaders reported improvements across several types of outcomes after adopting a performance 
contract: academic, staff, school climate, parent engagement, and financial. Leaders in 24 percent of districts 
perceived that academic outcomes, including student achievement, attendance, and graduation rates, improved 
substantially, and leaders in 55 percent of districts perceived that academic outcomes improved somewhat 
(figure 6). Similarly, leaders in 19-23 percent of districts perceived that staff, school climate, and financial out- 
comes improved substantially, and leaders in 47-53 percent of districts perceived that these outcomes improved 
somewhat. Leaders in a smaller proportion of districts perceived changes in parent engagement outcomes, with 
11 percent perceiving that parent engagement improved substantially, 43 percent perceiving that it improved 
somewhat, and 46 percent perceiving that it remained the same. 


District leaders more frequently cited staff and school climate outcomes than academic outcomes as 
benefiting from priority innovations 


District leaders identified changes to their professional learning approach as the most important contributor to 
improved outcomes after performance contract adoption. In surveys, district leaders reported that academic, 
staff, school climate, parent engagement, and financial outcomes changed after their district adopted a perfor- 
mance contract. Leaders in 35 percent of districts identified changes to their professional learning approach as 
the most important contributor to improved staff outcomes, 14 percent identified it as the most important con- 
tributor to improved school climate outcomes, and 3 percent identified it as the most important contributor to 
improved academic outcomes (figure 7). District leaders identified innovations that contributed to staff outcomes 
more frequently than innovations that contributed to school climate or student academic outcomes. 


7. The Georgia calls these waivers the “big four” because districts were required to request at least one of them in their SWSS applica- 
tions; Charter Systems automatically received all waivers. 
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Figure 5. After adopting a performance contract, many districts prioritized innovations that did not require a 
waiver, according to a survey of Georgia district leaders, 2019 
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Note: Includes innovative practices identified by leaders in at least 10 percent of districts as a priority for implementation. The number of districts is 
between 9 and 97, depending the innovation. 


a. Quality Basic Education is Georgia’s school funding formula, which determines the cost per pupil to provide an adequate education. 


Source: Authors’ analysis using data from surveys of district leaders administered by the Georgia Department of Education, 2019. 
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Figure 6. District leaders perceived improvements across broad outcomes after adopting a performance 
contract, according to a survey of Georgia district leaders, 2019 
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Source: Authors’ analysis using data from surveys of district leaders administered by the Georgia Department of Education, 2019. 


Figure 7. Innovations were most frequently perceived as benefiting staff and school climate outcomes, 
according to a survey of Georgia district leaders, 2019 


™ Academic outcomes & Staff outcomes ™ School climate outcomes 


Professional learning approach 


State, local, and federal funds in 
support of school improvement 


Number of school days, daily 
school hours, or school year 


School climate management program 
Course scheduling 

Student discipline 

Instructional spending 


Certification requirements 


Formative assessments used 
to guide instruction 


Curriculum vendors 


Personalized learning for students 


Programs focused on improving 
college and career readiness 


College and career academies 


Online or blended curricula 


oe a 


Safety programs 


jo) 
= 
2) 
N 
[o) 
w 
[o) 
p 
jo) 


Percent of districts stating that a waiver benefited 
academic, staff, or school climate outcomes 


Note: Sample includes 133 districts. 


Source: Authors’ analysis using data from surveys of district leaders administered by the Georgia Department of Education, 2019. 
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Changes in student achievement after districts adopted a performance contract were largely 
unrelated to priority innovations identified by district leaders, except the differentiated salary 
innovation and the alternative education programs innovation 


Most of the innovations included in the analysis were not related to changes in student achievement across 
grades and subjects after districts adopted a performance contract (table 3). However, two innovations were 
significantly associated with changes in achievement: achievement in grade 3-8 math was about .07 standard 
deviation higher for districts that prioritized innovations related to differentiated salary than for districts that 
did not, and achievement in grade 9 English was about .07 standard deviation lower for districts that prioritized 
innovations related to alternative education programs than for districts that did not. 


Variation in how long districts have had a performance contract is important in contextualizing these findings. 
Most districts have prioritized innovations under their performance contract for one or two years, which may not 
be enough time to reveal a relationship with changes in student achievement. In addition, not all prioritized inno- 
vations have been fully implemented, which also may limit the relationship between prioritized innovations and 
changes in student achievement. In survey responses, 48 percent of district leaders stated that innovations were 
being implemented consistently across schools, 48 percent of district leaders stated that innovations were being 
implemented differently across schools, and 1 percent stated that innovations were not yet being implemented in 
any schools.® 


Table 3. Changes in student achievement and their relationship with the innovations Georgia districts 
prioritized under their performance contracts, 2005/06-2017/18 


Grades 3-8 Grades 9-12 


English end-of- Algebra | end-of- 
NETaE Lo) (<1 English language arts course assessment course assessment 


Priority academic innovations 


Alternative education programs —0.03 (0.03) —-0.02 (0.03) -0.07 (0.03)* —0.03 (0.05) 
College and career academies 0.00 (0.02) —0.03 (0.03) —0.03 (0.03) -0.05 (0.04) 
Personalized instruction 0.01 (0.04) 0.02 (0.04) 0.01 (0.04) 0.08 (0.08) 
School climate management 0.01 (0.18) 0.20 (0.20) 0.22 (0.22) —-0.12 (0.38) 
Priority human resources innovations 

Certification —0.02 (0.02) —0.02 (0.02) 0.04 (0.03) —0.01 (0.04) 
Differentiated salary 0.03 (0.03) 0.07 (0.03)* 0.04 (0.03) 0.06 (0.06) 
Class size 0.01 (0.02) 0.03 (0.02) —0.00 (0.02) —0.01 (0.04) 
Professional learning -0.05 (0.03) —0.02 (0.03) 0.02 (0.03) 0.02 (0.06) 
Priority financial innovations 

Instructional spending 0.01 (0.02) —0.00 (0.03) —-0.00 (0.03) —0.03 (0.04) 
Noninstructional spending 0.02 (0.02) 0.00 (0.03) 0.01 (0.03) —0.02 (0.04) 
Number of districts 133 133 133 133 
District-by-year observations 1,596 1,596 1,584 1,395 


* Significant at the .05 level or lower. 
Note: Numbers in parentheses are standard errors. Models include controls for student and district characteristics, including adoption cohort group. 


Source: Authors’ analysis using administrative data from Georgia Department of Education, 2005/06-2017/18. 


8. About 2 percent of district leaders did not respond to this question. 
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Limitations 
This study has four main limitations. 


The findings related to changes in student achievement after districts adopted a performance contract (research 
questions 1 and 3) are driven heavily by districts that have implemented performance contract—related innova- 
tions for only one or two years. It will be important to continue to track changes in achievement as more districts 
in the early adoption cohort group approach five years with a performance contract. 


Survey (self-report) data, such as those used in this study, present several limitations. First, the data depend on 
respondent accuracy; in some cases, respondents may not provide accurate responses and reflections or may not use 
the survey as intended. For example, district leaders might be more likely to indicate prioritizing innovations that they 
were first exposed to in the online survey (see appendix B for further discussion). Second, survey nonresponse can 
introduce bias. In this study the district-level response rate was 75 percent, which is below the 85 percent response 
goal. Analyses for potential response bias indicated that the sample of districts with responses was representative of 
all districts in the state on observable characteristics (see table B3 in appendix B). Still, because these observable vari- 
ables were not strongly correlated with the prioritized innovations, it is possible that the districts that responded do 
not fully represent the state. Results should therefore be interpreted with caution. This limitation also applies to find- 
ings related to the analysis of the relationship between changes in student achievement and innovations that district 
leaders reported prioritizing in the survey (research question 3). Third, the priority innovations identified in the survey 
were based on an item asking respondents if they had implemented or planned to implement performance contract— 
related innovations. Some or many of the districts might still be in the planning phase of innovations, so associations 
with changes in achievement would not be expected. In addition, districts might have been implementing or prioritiz- 
ing at a substantial level other innovations that district leaders did not list as top priorities. This reinforces the need for 
additional research into school-level changes in practice after districts adopt a performance contract. 


Third, the qualitative findings are based on a small number of interviews with leaders in 10 districts that were 
among those that experienced the largest improvement in student achievement after adopting a performance 
contract. Their experiences are unlikely to be representative of all districts, but they were chosen for their poten- 
tial to provide insight into potentially promising local changes in practice that other districts might find helpful 
when implementing innovations under their own performance contract. 


Fourth, the study is not intended to determine cause and effect. Finding an association between an innovation 
and a change in student achievement, even when statistically significant, does not mean that the innovation 
caused the change and should not be interpreted as such. The study’s findings reveal only the strength of the 
associations between specific indicators and outcomes. These associations can be used to identify potential suc- 
cesses and challenges and provide guidance to state education administrators on areas that show promise or 
require additional support for districts. 


Implications 


The study findings suggest potential directions for the Georgia Department of Education to support districts’ 
efforts to implement innovations associated with the waivers in their performance contracts and further under- 
stand districts’ experiences under the flexibility policy. 


The Georgia Department of Education might consider monitoring whether statewide changes in student achievement 
emerge over the coming years as performance contracts and related innovations mature. The study found minimal 
evidence of relationships between innovations and changes in achievement after districts adopted a performance 
contract. The districts may not have fully implemented performance contract—related innovations, especially because 
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most districts have had their performance contracts for only one or two years. It is not possible to know whether 
districts have started implementing all their prioritized innovations or whether innovations have only been planned. 
In time, the relationship between innovations and longer-term changes in achievement should become clearer. 


Because the study found improvements in student achievement in English language arts and math in some districts, 
despite minimal evidence of changes in achievement statewide, the Georgia Department of Education might con- 
sider encouraging districts to collect systematic data on the innovations they implement under their performance 
contract. Doing so might help increase understanding of how changes in practice are associated with changes in 
student, staff, and other outcomes. This study relied on district leaders’ historical accounts of changes in local pri- 
orities and innovations related to their performance contracts, which have several limitations, as discussed in the 
previous section. Regularly reviewing changes in district policies and practice that relate directly to performance 
contracts could help build a robust database to drive continuous improvement. Indeed, the Georgia Department of 
Education calculates whether districts met their performance targets each year, but a review that extends beyond 
accountability could provide interim information on whether and how districts are implementing innovations. 


In addition to monitoring changes in academic achievement over the next three to four years, the Georgia Depart- 
ment of Education might consider examining the relationship between adopting a performance contract and staff 
retention and mobility, especially because many district leaders perceived benefits to school climate and staff 
outcomes. One specific question for further study is whether using teacher certification flexibility can help dis- 
tricts resolve challenges in attracting and retaining teachers, especially in small rural districts. About 71 percent of 
districts reported prioritizing flexibility related to teacher certification so that they could recruit and hire teachers 
who may not be certified in order to fill vacant positions in particular subject areas. The department also might 
want to further examine how this practice is associated with student achievement and whether achievement 
differs for students of noncertified teachers in certain subject areas or grade levels. 


The findings also suggest a need to conduct research on implementation of the flexibility policy at the school 
level. Such research could help the Georgia Department of Education better understand the extent to which 
priorities at the district level successfully translate into changes in practice at the school level. Although the lack 
of clear evidence on the relationship between priority innovations and changes in student achievement may be 
driven by the relative immaturity of most performance contracts, those innovations may have more precise man- 
ifestations at the school level that better explain variation in changes. Regional Educational Laboratory South- 
east researchers are partnering with the department to conduct a follow-up study that examines differences in 
changes in achievement among schools within districts after performance contract adoption as well as schools’ 
experiences implementing innovations related to their districts’ waivers. 
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